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COMPUTER  SYSTEM  COMMAND 


I .  Background 

The  U.S.  Army  Institute  for  Research  in  Management  Information  and 
Computer  Science  (AIRMICS)  entered  into  an  agreement  with  the  Georgia 
Tech  Research  Institute  to  fund  a  three  month  study  titled,  "Modification 
to  Existing  U.S.  Army  Software  Micro-Resource  Estimation  Procedure".  The 
study  was  conducted  by  Dr.  Gerald  J.  Thuesen  of  the  School  of  Industrial 
and  Systems  Engineering. 

The  setting  for  the  study  was  the  Computer  Systems  Command  Support 
\  * 

Group  at  Fort  Lee,  Virginia. ^(Support  Group  Lee)  This  commandos  within 
the  U.S.  Army  Computer  Systems  Command  which  is  responsible  for  the 
design,  development  and  maintenance  of  management  informations  systems 
for  over  200  military  computer  installations  throughout  the  world. 

Although  organizationally  separated, Support  Group  Lee  is  responsible 
for  all  maintenance  of  computer  programs  utilized  by  the  Army  Logistics 
Command  also  located  at  Fort  Lee.  The  Army  Logistics  command  refered  to 
as  a  proponent  agency  prepares  and  submitts  requests  for  program  modifi¬ 
cations  to  Support  Group  Lee.  These  requests  are  then  routed  to  the  divi¬ 
sion  within  Support  Group  Lee  that  is  responsible  for  the  programs  affected. 

To  manage  the  change  request  from  its  receipt  to  final  disposition  it 
is  necessary  to  estimate  the  resources  that  will  be  required  to  make  the 
modifications  requested.  These  estimates  are  usually  prepared  by  the  systems 
analyst  or  programmer  who  is  familiar  with  the  affected  programs.  Then  the 
individual  requests  are  grouped  into  System  Change  Packages  (SCP's)  with 
the  approval  of  the  proponent  agencies.  These  SCP's  are  then  the  basis  for 
commitments  to  the  proponent  agency  regarding  the  time  to  completion.  Also, 
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the  SCP 1 s  form  the  basis  for  controlling  and  managing  within  Support  Group 
Lee  the  resources  necessary  to  perform  the  modifications. 

The  present  method  of  resource  estimation  for  the  individual  change 
request  uses  a  micro-estimating  technique  that  is  embodied  in  the  USACSC 
Form  50.  (See  Appendix  A)  The  procedure  utilized  on  this  form  requires 
numerous  decisions  regarding  program  complexity*  the  quality  and  avail¬ 
ability  of  resources,  knowledge  required  and  the  extent  of  the  change. 

In  addition,  predetermined  factors  are  used  to  account  for  indirect  time 
related  to  the  task.  The  result  of  selecting  various  values  for  each 
element  on  the  form  and  combining  these  elements  as  described  yields  an 
estimate  in  man-days  of  the  time  to  do  the  job.  This  estimate  provides 
the  elapsed  time  to  accomplish  the  job  rather  than  the  direct  time  that 
would  be  expended  executing  the  actual  work. 

Raven  Systems  and  Research,  Inc.  was  assigned  the  task  to  study  the 
current  estimating  procedure  based  on  the  Form  50.  Their  conclusions  and 
recommendations  are  included  in  their  August  1979  report  to  AIRH1CS  (2  ). 
Essentially,  the  report  discusses  how  the  users  view  Form  50  and  its 
resulting  estimates,  the  accuracy  of  those  estimates,  and  the  difficulties 
of  varifying  those  estimates. 

Since  the  interval  of  the  Raven  Systems  study  overlapped  the  initia¬ 
tion  of  this  study.  Dr.  Thuesen  worked  directly  with  the  participants  in 
the  Raven  System  study.  Although  most  of  the  data  gathering  had  been 
accomplished.  Dr.  Thuesen  and  Dr.  Ronald  Askin  participated  in  the  design 
of  the  modified  Form  50  proposed  in  the  Raven  Systems  report.  Although 
the  basic  approach  was  not  changed,  the  procedures  were  considerably 
simplified  to  at  least  reduce  the  computational  errors  being  introduced 
by  the  more  complex  Form  50. 
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The-  task  of  this  study  is  to  utilize  the  data  gathered  by  Raven 
Systems  and  to  develop  an  improved  method  of  micro-resource  estimation. 

That  is,  we  would  investigate  whether  an  improved  estimating  methodology 
could  be  developed  so  that  better  resource  planning  and  control  would  be 
possible.  This  new  methodology  would  hopefully  replace  the  present  approach 
represented  by  Form  50  and  expand  the  application  of  the  estimates.  A 
major  difficulty  pointed  out  by  the  Raven  Systems  study  was  the  inability 
to  elicit  the  actual  man-days  to  perform  a  particular  SCR.  Therefore, 
any  approach  utilized  in  this  study  could  not  be  based  on  any  comparisons 
of  actual  versus  estimated  data.  Therefore,  this  study  not  only  proposes 
a  new  method  of  estimating,  but  also  a  procedure  for  validating  the  new 
approach  that  is  suggested.  This  validation  would  compare  on  a  statistical 
basis  actual  time  with  estimated  time. 

it.  Procedures 

The  procedures  utilized  in  this  study  included  two  phases.  The  first 
phase  is  primarily  concerned  with  understanding  how  Support  Group  Lee 
tune i  io-.j;  and  familiarization  with  the  literature  that  seemed  applicable 
to  the  problem  of  estimating  resources.  Dr.  Thuesen  and  Dr.  Askin  attended 
meetings  within  AIRMICS  at  Georgia  Tech  for  orientation  and  considerable 
time  was  spent  with  James  Gantt  of  AIRMICS  in  discussing  how  Support  Group 
Lee  operated.  Then,  as  Raven  Systems  finished  their  interviews,  we  held 
approximately  five  meetings  with  their  personnel  to  get  a  more  detailed 
description  of  how  the  micro-estimating  procedure  was  presently  operating. 

At  the  same  time.  Dr.  Thue  an  and  Dr.  Askin  were  investigating  the 
literature  with  regard  to  techniques  that  would  be  applicable  to  the 
micro-estimating  problem  faced  by  Support  Group  Lee.  Because  almost  all 
the  workload  at  Support  Group  Lee  consists  of  modification  of  computer 
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programs  rather  than  program  development,  an  attempt  was  made  to  find 
existing  methods  that  would  be  applicable.  One  document  that  describes 
in  detail  the  present  methodologies  for  resource  estimation  is  a  1978 
report  by  H.  M.  Wadsworth  (3  ).  Unfortunately,  none  of  the  methods 
discussed  in  this  report  seemed  applicable  to  the  resource  estimation  of 
computer  program  maintenance. 

The  second  phase  of  this  study  was  focused  on  developing  a  new  approach 
for  micro-estimating  and  determining  the  method  of  validation  that  should 
be  followed.  This  phase  required  two  trips  for  interviews  with  personnel 
in  Support  Group  Lee.  The  first  trip  was  used  to  determine  whether  the 
modified  Form  50  proposed  by  Raven  Systems  would  be  sufficient.  After  the 
interviews,  it  was  clear  that  an  approach  that  estimated  direct  man-days 
rather  than  elapsed  man-days  would  be  more  useful  for  control  and  management 
of  the  SCP's. 

The  second  trip  was  utilized  to  elicit  suggestions  and  comments  con¬ 
cerning  the  new  approach  that  had  been  developed  since  our  first  contact 
with  Support  Group  Lee.  Both  project  leaders  and  programmers  were  shown 
two  different  forms  based  on  the  same  basic  approach.  The  form  in  Appendix 
B,  which  was  finally  adopted,  requires  considerably  less  detailed  esti¬ 
mating  than  the  form  shown  in  Appendix  C.  The  more  detailed  procedure 
shown  in  Appendix  C  was  generally  rejected  by  those  who  would  have  been 
responsible  for  executing  the  estimating  procedure.  The  concensus  opinion 
was  that  it  is  not  realistic  to  separate  the  various  inputs  into  such 
small  elements.  Thus,  no  improvement  in  the  final  estimate  would  be 
achieved  by  making  the  additional  effort  to  work  with  the  larger  number 
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In  addition,  discussions  concerning  the  best  approach  for  validating 
the  new  estimating  procedure  were  had  with  those  who  would  be  directly 
involved  in  any  new  estimating  methodology.  It  was  concluded  that  the 
basic  estimating  unit  should  be  each  system  change  request.  (SCR)  By 
recording  the  estimates  made  for  each  SCR  before  the  work  is  performed 
and  utilizing  the  project  management  system  (PMS)  to  record  the  actual 
effort  expended,  data  for  validation  would  be  available  for  the  first 
t  ime. 

The  data  resulting  from  this  data  collection  activity  would  then 
be  analyzed  to  test  for  significant  statistical  deviations  between  the 
actual  and  estimated  values.  The  statistical  tests  to  be  applied  have 
been  determined  as  part  of  this  study  and  they  will  be  discussed  in  detail. 

The  final  task  in  the  second  phase  of  this  investigation  will  be  the 
preparation  of  the  final  report. 

III.  Analysis 

Before  the  initial  trip  to  interview  personnel  for  this  study, 

Dr.  Thuesen  and  Dr.  Askin  held  four  lengthy  sessions  with  Robert  Barrier 
and  Gaye  Stewart  of  Raven  Systems.  The  purpose  of  these  meetings  were  to 
gather  information  pertinent  to  the  study  to  be  undertaken.  Since  it  was 
known  that  the  final  report  to  be  prepared  by  Raven  Systems  would  not  be 
completed  by  the  time  our  study  was  initiated,  we  worked  informally  to 
understand  the  status  of  the  micro-estimation  procedure  that  was  currently 
(n  operation  at  Support  Group  Lee. 

A  number  of  important  issues  were  identified  and  it  is  these  issues 
that  became  the  basis  for  our  proposed  solution  to  the  micro-estimating 
process.  It  became  evident  that  any  micro-estimating  procedure  must 
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provide  information  that  was  meaningful  for  aggregating  the  change  requests 
into  the  System  Change  Packages.  (SCP)  This  process  described  as  a  "scrub" 
session  allowed  the  proponent  agency  and  the  personnel  at  Support  Group  Lee 
to  prioritize  the  change  requests  which  were  to  be  included  in  the  new  SCP'-. 
At  this  time,  the  estimates  of  man-days  determined  by  the  Form  50's 
are  utilized  to  provide  some  indication  of  the  resources  that  would  be 
required  by  Support  Group  Lee  to  accomplish  a  particular  job.  The  estimates 
that  were  used  in  this  process  were  sometimes  modifications  of  the  Net 
Development  Time  as  shown  on  the  Form  50.  These  modifications  were  based 
on  a  reduction  of  the  Net  Development  Time  obtained  by  multiplying  by  fac¬ 
tors  of  0.4  to  0.5.  These  factors  based  on  the  experience  of  team  leaders  at 
Support  Group  Lee,  gave  us  a  strong  indication  that  those  team  leaders  who, 
according  to  the  Raven  Systems  people,  were  more  sophisticated  in  their 
management  techniques  realized  that  the  Form  50  estimate  was  not  accurate. 

Because  we  had  no  data  on  which  to  verify  these  opinions,  we  estab¬ 
lished  the  position  that  team  leader  experience  will  approximate  the  actual 
outcome  with  a  good  degree  of  confidence.  This  conclusion  is  based  on  the 
consulting  experience  of  Dr.  Thuesen,  who  has  observed  numerous  instances 
where  approximate  methods  based  on  experience  usually  provides  results  close 
to  what  might  have  been  achieved  with  more  systematic  analysis,  (i.e.  Common 
sense  tempered  with  experience  leads  to  near  optimal  solutions) 

Based  on  these  observations,  it  was  concluded  that  the  Form  50,  which 
provided  total  elapse  time,  was  not  the  appropriate  information  for  estab¬ 
lishing  change  request  priorities  in  the  "scrub"  session.  It  was  the  direct 
hours  associated  with  the  change  requests  that  would  be  more  useful  in  this 
planning  activity. 
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Once  an  SCP  is  assembled,  the  other  important  function  that  is  depen¬ 
dent  on  good  micro-estimating  is  project  control  or  project  management. 

The  SCP  becomes  the  basic  unit  for  determining  milestones  and  final  deli¬ 
very  dates.  According  to  the  Raven  Systems  people,  more  preestablished 
milestones  for  SCR's  are  missed  than  are  achieved.  Some  of  these  mile¬ 
stones  have  been  missed  by  more  than  200/’.  It  was  our  impression  based  on  con¬ 
versations  with  Raven  Systems  personnel  and  Support  Group  Lee  personnel, 
that  this  inability  to  meet  stated  deadlines  was  the  most  serious  problem 
faced . 

Accepting  this  premise  then  caused  us  to  examine  how  improved  control 
of  the  SCP's  could  be  accomplished.  In  our  discussions  with  Raven  Systems 
and  Support  Group  Lee,  it  was  recognized  that  the  problem  of  project  control 
bad  many  facets;  one  of  which  was  the  lack  of  accurate  time  estimates  for  the 
SCP's.  However,  the  most  serious  problem  seemed  to  be  related  to  the  wide  vari- 
etv  nr  Jack  of  methods  used  by  the  various  project  leaders  for  project  control 
and  resource  planning.  Although  the  Projec  Management  System  (PMS)  is 
available  and  encouragement  is  given  by  upper  management  for  its  use,  only 
a  few  individuals  were,  in  fact,  using  this  system.  Since  PMS  is  designed 
with  great  user  flexibility,  each  user  has  tailored  his  own  individual 
reporting  scheme  and  what  use  is  occuring  is  non-standardized. 

Because  PMS  is  a  powerful  management  tool  and  because  an  even  more 
useful  Decision  Support  System  is  currently  under  development  by  AIRMICS, 
we  concluded  that  any  micro- resource  estimating  technique  that  is  developed 
should  provide  the  information  in  a  form  that  increases  the  efficiency  of 
these  management  tools.  The  data  form  which  allows  greatest  utilization 
of  PMS  is  to  input  direct  man-days  for  task  completion.  Direct  man-days 
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represent  the  estimate  of  actual  hours  to  be  spent  on  Review  and  Analysis, 
Design,  Communication,  Coding,  Level  I  testing  and  Documentation.  Addi¬ 
tional  data  regarding  the  future  availability  of  resources  due  to  planned 
vacations,  scheduled  leaves,  and  other  anticipated  manpower-reductions  are 
then  input  directly  into  PMS .  This  approach  allows  each  unit  to  plan 
more  realisticly  according  to  their  specific  resource  availabilities. 

The  approach  on  which  the  Form  50  is  based  provides  total  elapsed 
time  that  is  determined  by  using  constant  factors  for  the  estimation  of 
resources  required  by  the  non-direct  activities.  As  a  result,  each  unit 
has  less  ability  to  recognize  their  own  peculiarities  regarding  their 
non-direct  resource  requirements.  In  addition,  there  was  no  indication 
that  the  Form  50  methodology  would  provide  resonable  estimates  of  the 
direct  time.  Therefore,  it  was  concluded  that  a  new  method  needed  to  be 
developed  that  would  focus  on  micro-resources  estimation  of  direct  man¬ 
power  requirements. 

IV.  Results 

Working  from  the  basic  premise  that  the  appropriate  information  to  be 
provided  by  any  micro-estimating  procedure,  should  be  in  terms  of  direct 
effort  expended,  an  entirely  new  estimating  procedure  was  developed.  This 
procedure  requires  that  estimates  be  made  in  such  a  manner  that  each  indi¬ 
vidual  contribution  to  the  direct  work  effort  performed  be  included. 

Because  the  estimates  are  to  be  in  direct  man-days,  the  estimates  for  each 
of  the  component  activities  required  to  complete  the  SCR  will  be  additive. 
Thus,  a  rather  simple  arithmetic  procedure  has  evolved  which  will  be  useful 
in  the  estimation  of  resource  requirements  for  both  the  SCR's  and  the  SCP's 
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\.  I'.sl  im.it  in^  t Ik*  SCR 

Because  the  most  basic  work  task  is  the  program  affected  in  each 
system  change  requesL  (SCR),  it  was  decided  that  this  basic  unit  should 
he  the  basis  lor  micro-resource  estimating.  With  the  possibility  of  one 
or  more  individuals  being  directly  involved  in  the  work  activity,  the 
decision  was  made  that  a  direct  man-day  estimate,  for  each  individual  be 
recorded.  Thus,  if  the  nature  of  the  work  indicates  that  a  systems 
analyst  and  two  programmers  will  be  involved,  three  separate  estimates 
are  to  ho  made.  Each  estimate  should  be  produced  by  each  of  the  individuals 
in  light  of  the  tasks  that  each  anticipates  being  required  to  perform.  If 
this  level  of  knowledge  about  who  is  to  perform  what  specific  tasks  is  not 
available,  then  a  single  estimate  will  be  made.  The  general  rule  is  that 
as  many  estimates  will  be  made  for  each  program  within  a  SCR  as  there  are 
individuals  who  are  actually  required  to  make  inputs  regarding  their  esti¬ 
mated  functional  involvement  in  performing  work  on  that  particular  pro¬ 
gram. 

The  form  presented  in  Appendix  B  will  be  the  mechanism  for  recording 
the  estimates  of  the  individuals  involved.  Thus,  not  only  must  the  system, 

SCK  1 1 1 mil >•  ■  i  ,  and  program  affected  he  identified,  hut  each  contributor  to 
the  overall  estimate  must  also  he  known. 

Alter  eons iderab le  study  and  review  by  personnel  at  Support  Group  Lee, 
six  important  categories  of  work  effort  were  identified.  These  six  categories 
encompass  the  significantly  different  types  of  work  activities  that  are 
usually  associated  with  a  systems  change  request  (SCR).  Because  it  was 
recognized  that  the  individuals  that  impact  on  a  SCR  may  be  performing 
different  functions,  it  is  important  that  each  category  be  interpreted  with 
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respect  to  each  individuals  contribution.  That  is,  a  systems  analyst  may 
view  the  work  effort  required  for  the  Review  and  Analysis  category  quite 
differently  than  a  programmer  would  view  the  same  category.  The  systems 
analyst  may  hi'  reviewing  the  system  of  prog, rams  being.  .  1 1  I  ■  t  >  -<  I  while  I  In 
programmer  would  he  spending  his  review  effort  In  under  st  and  I  up,  how  l  lie 
specific  program  for  which  he  is  responsible  would  be  changed. 

B.  Estimating  Categories 

The  six  estimating  categories  on  which  the  proposed  micro-resource 
estimating  procedure  is  based  are: 

Review  and  Analysis 
Design 

Communication 

Coding 

Level  I  Testing 
Documentation 

Each  individual  involved  in  the  estimating  procedure  should  make  estimates 
only  for  those  categories  in  which  a  direct  contribution  to  the  completion 
of  the  task  will  be  made.  For  each  category  estimated  by  an  individual,  a 
single  value  representing  the  anticipated  direct  man-days  require  will  be 
recorded.  If  there  are  categories  for  which  an  individual  has  no  involve¬ 
ment,  his  estimate  should  be  left  blank  for  that  category.  It  is  antici¬ 
pated  that  a  Systems  analyst  will  not  normally  contribute  in  the  Coding 
category,  so  no  entry  should  he  made. 

It  should  be  pointed  out  that  activities  such  as  Level  II  Testing, 
Level  III  Testing,  Environmental  Test  and  Field  Validation  are  considering 
outside  the  proposed  estimating  procedure.  Because  these  activities  are 
generally  handled  differently  and  they  are  much  more  dependent  on  avail¬ 
ability  of  computer  time,  it  Is  believed  that  a  different  approach  must  he 
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followed  in  order  to  develop  accurate  estimates  for  these  activities.  How¬ 
ever,  it  Is  important  that  accurate  estimates  of  these  activities  need  to 
he  available  if  sound  planning  and  control  is  to  be  achieved  at  Support 
Croup  l.ee. 

1.  Review  and  Analysis 

In  general,  this  activity  represents  all  the  preparatory  work  that  is 
necessary  before  any  decision  regarding  implementation  of  the  change  request 
can  he  made.  Fur  the  System  Anslyst,  this  would  include  the  effort  spent 
co-ordinating  and  communicating  with  the  proponent  agency  to  insure  there  is 
an  accurate  understand ing  of  the  request.  In  addition,  time  spent  studying 
the  relation  of  the  change  request  to  the  system  of  programs  affected  would 
hi'  included  in  this  category. 

On  the  other  hand,  this  category  represents  for  the  programmer  the 
effort  to  he  expended  in  understanding  the  program  files  and  logic  to  be 
affected  hv  the  change  request.  This  estimate  should  include  studying 
program  listings,  locating  documentation,  recording  the  nature  of  the 
change  request,  and  understanding  the  scope  of  the  problem. 

1.  Design 

This  category  is  concerned  with  the  effort  that  is  to  hi*  expended  in 
translating  the  change  request  into  specific  remedies  that  will  insure 
successful  completion  of  the  task.  Primarily,  this  category  will  recog¬ 
nize  the  time  involved  in  synthesis,  and  the  development  of  specific  change 
specifications.  It  is  anticipated  tiiat  both  system  analysts  and  programmers 
will  he  involved  in  this  type  of  activity.  The  systems  analyst  will  be 
involved  with  design  regarding  how  the  change  request  affects  the  svstem 
of  programs  while  flu.'  programmer  will  be  concerned  with  how  the  change 
request  affects  his  particular  program. 
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3 .  Coramun ic at  ion 

The  effort  that  arises  from  the  interaction  between  the  system  analyst  an.! 
programmer  to  assure  that  the  changes  to  each  of  the  system  parts  will  lea. I  t<> 
the  overall  objective  of  the  change  request  will  be  recorded  as  communication. 
This  activities  will  include  both  written  and  verbal  communication.  When 
this  type  of  activity  occurs,  the  systems  analyst  should  estimate  hit;  fitvolv.  - 
ment  (he  may  be  involved  with  a  number  of  programmers  for  a  particular  St'.K) 
and  the  programmer  will  estimate  his  effort  in  this  activity. 

4 .  Cod  ij\i» 

Under  this  category  is  recorded  all  the  effort  expended  by  the  pro¬ 
grammer  to  translate  the  change  specifications  into  coded  logic.  Given 
the  change  specifications,  what  coding,  file  manipulation,  and  other  pro¬ 
gramming  activities  are  necessary  to  produce  a  source  deck  that  is  ready 
for  testing. 
r) .  Level_I_J e s  t  ing 

This  category  reflects  all  the  effort  that  is  required  to  test  the 
program  that  has  boon  modified  to  satisfy  a  change  request  so  that  the  pro¬ 
gram  will  he  ready  lor  bevel  II  Testing.  The  effort  recorded  should  reflect 
activities  which  include  compilation,  development  of  test  data,  analysis  of 
test  results  and  corrections.  It  Is  important  for  this  category  that  only 
direct  effort  expended  by  considered  so  that  delays  due  to  machine  avail¬ 
ability  are  not  part  of  the  estimate.  Machine  dependent  delays  should  be 
considered  as  indirect  effort  that  is  required  to  satisfy  the  change  request. 

6 .  Document at  ion 

Any  documentation  effort  associated  with  a  completed  change  request  will  be 
recorded  here.  Documentation  includes  any  changes  to  operating  or  system 
manuals  and  any  documentation  associated  with  changes  in  program  logic. 

Again,  only  the  direct  effort  expended  by  the  system  analysis  or  programmer 
shwuld  be  considered  here. 

/  -  —  - - 

/ 
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In  addition  to  liio  information  that  appears  on  tin’  front  of  the 
Direct  Man-Day  Estimating  Form,  our  discussions  with  Support  Croup  Lee 
personnel  revealed  that  some  written  description  of  the  nature  of  the 
changes  that  were  the  basis  for  Lite  estimate  would  be  extremely  useful. 
Because  the  original  estimate  for  the  SCR  is  made  months  in  advance  of 
the  inclusion  ot  that  SCR  in  a  SCP,  it  was  usual  that  the  person  making 
tiie  estimate  was  not  the  same  person  actually  implementing  the  change 
retpiest.  Ail  personnel  we  questioned  insisted  that  if  the  person  making 
the  original  estimate  would  record  in  writing  a  description  of  the  program 
changes  necessary  to  satisfy  the  change  request,  much  greater  under¬ 
standing  of  the  estimates  would  be  possible.  Also,  the  written  record 
would  provide  a  continuity  so  that  the  person  who  must  actually 
implement  the  change  request  will  not  have  to  begin  his  analysis  from 
scratch. 

This  narrative  description  of  anticipated  changes  includes  a  place 
to  list  the  files  that  would  he  affected  by  the  SCR.  Again,  this  informa- 
I  ion  is  intended  to  help  validate  the  estimate  and  to  provide  continuity 
to  tin.'  islimat  ing  process. 

C.  Estimating  the  Systems  Changty  Package  (SCP) 

The  individual  estimates  on  the  SCR's  are  intended  to  be  the  basis 
for  determining  delivery  times  for  a  SCP.  It  is  anticipated  that  following 
the  "scrub"  session  milestone  estimates  would  be  developed  by  combining 
the  direct  man-day  estimates  for  the  SCR's  contained  in  a  SCP  with  the 
indirect  man-days  estimated  for  all  personnel  activities  not  directly 
a l I r i bill  ah  1 e  to  the  change  request.  These  estimates  arc  then  input  to  the 
Project  Management  System  (PMS)  so  that  the  total  resources  can  he  scheduled 
.uni  reasonable  milestones  for  completion  of  system  change  packages  can  he 
I i xed .  This  process  should  he  relatively  straight  forward  as  the  direct 


man-day  estimates  resulting  for  the  new  form  will  be  additive  with  the 
Indirect  man-day  estimates. 

D .  Testing  the  Validity  of  Estimating  Technique 

The  validation  of  the  micro-resource  estimating  technic|in'  proposed 
in  this  study  should  have  two  distinct  benefits.  First,  the  osl  im.il  e  . 
made  prior  to  the  initiation  of  t  lie  work  can  he  st  a  t  i  s  t  i  ea  I  I  v  eoiupa  i  ed 

with  the  actual  effort  recorded  lor  that  work.  For  each  ot  these  e.ite 

gorles  a  statistical  comparison  will  be  made  to  see  how  well  each  of 
these  type  of  estimates  can  be  made.  Also,  the  total  man-days  estimated 
will  be  compared  with  the  actual  total  effort  expended  so  that  the  pro¬ 
cess  of  summing  the  categories  to  find  aggregate  estimates  can  be  validated 
The  second  important  effect  of  recording  estimated  and  actual  effort 

expended  is  the  increased  awareness  that  will  be  engendered  in  the  Support 

Croup  Lee  personnel.  This  awareness  should  encourage  more  serious  consider 
t ion  of  the  individual's  role  in  making  reliable  estimates  when  using  the 
proposed  procedure.  Many  of  the  persons  interviewed  felt  that  this  would 
be  one  of  the  most  positive  outcomes  of  this  study. 

1 .  Test  Procedures 

T^  shall  represent  the  time  (or  the  form  initiated  at  that  time)  at 
which  an  SCR  is  first  obtained  from  the  proponent  agency. 

T 2  shall  represent  the  time  (or  the  form  recorded  at  that  time)  immedi 
ately  before  the  scrub  session.  (It  is  assumed  that  at  T^  the  person  or 
persons  who  will  be  working  on  the  SCR  can  be  identified) 

Tj  shall  represent  the  time  (or  data  recorded)  just  after  completion 
of  the  work  on  the  SCR.  (Actually,  as  categories  of  work  are  completed, 
the  effort  will  be  recorded  and  input  to  PMS) 
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2.  Data  Collection 

Two  teams,  SAAS  LVI.  [  headed  by  Mr.  K.  A.  DeLong  and  A(-)/ACS  Team 
by  Mr.  I.  T.  Putton,  have  been  selected  by  management  at  Support  Group  Lee 
to  provide  tlie  test  environment  for  the  proposed  micro-estimating  procedure. 
Both  groups  will  be  utilized  to  provide  estimated  and  actual  data  and  it 
is  intended  that  this  data  be  accumulated  for  approximately  one  year.  This 
sludv  period  will  allow  for  the  inclusion  of  3  to  4  completed  SCP's  for 
each  team.  Based  on  the  normal  size  of  an  SCP,  it  is  anticipated  that  150 
to  J3D  separate  estimates  will  be  required  providing  a  solid  data  base. 

I  ia  t  a  Co  N  eo  L_ ion  _S_L  eps 

a.  At  time  T^  any  involved  systems  analyst  and  programmer  will 
fill  out  separate  estimating  forms  for  each  program  affected 
by  the  SCR.  These  will  then  be  filed  together  as  a  combined 
estimate. 

b.  At  time  T ^  the  systems  analyst  and  any  programmers  who  are 
actually  responsible  for  implementing  the  SCR  will  fill  out 
separate  estimates  on  the  form.  There  will  be  a  form  required 
for  each  program  affected  by  the  SCR.  These  estimates  will 

be  filed  together  and  the  sum  of  the  individual  estimates 
will  represent  the  total  direct  time  estimated  for  the  SCR. 

c.  As  soon  as  the  actual  work  commences  on  the  SCR,  the  actual 
direct  time  expended  in  each  o'  the  six  categories  will  be 
recorded  daily.  These  daily  records  will  then  be  the  basis 
for  weekly  reporting  of  actual  effort  expended.  By  utilizing 
t  lie  Project  Management  System  (RMS)  the  weekly  totals  of 
actual  direct  mail-days  allocated  to  each  of  the  six  categories 
will  be  available  for  analysis.  (It  Is  important  that  the  same 
format  and  reportin'’  categories  he  used  in  PMS  by  all  participants 
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in  tin-  study)  At  tilin'  T  comp.i  r  I  .sons  »t  actual  cl  lol  l  vri  mis 
estimated  effort  wilL  be  made. 

3.  Test  of  Direct -Man  Day  Estimating  Form 

By  having  estimates  at  and  and  actual  values  at  T^,  there  are 
two  primary  premises  to  test.  Comparing  actual  effort  expended  to  the 
estimates  made  at  T2  we  have  a  test  of  estimating  proficiency  in  the  most 
favorable  of  circumstances.  That  is,  the  individuals  who  are  actually 
assigned  to  do  the  work  will  be  making  the  estimate  immediately  prior  to 
starting  the  job,  hence  the  same  Individual  will  be  making  the  est  (mate  and 
doing,  the  work.  It  is  assumed  that  an  individual  can  best  estimate  the 
effort  he  must  expend  tor  satisfactory  completion  of  the  job.  We  believe 
that  the  individual  knows  his  own  capabilities  better  than  someone  else. 

Tiie  second  premise  to  test  is  whether  estimates  made  at  T^  are  reli¬ 
able  estimates  of  the  actual  effort  that  is  expended.  In  this  case,  those 
who  are  making  the  estimates  are  not  necessarily  those  who  will  be  doing 
the  actual  work.  Also,  these  initial  estimates  usually  occur  two  to  six 
months  before  the  work  actually  begins  and,  therefore,  there  may  not  be  a 
clear  definition  of  the  actual  tasks  that  must  be  accomplished. 

V.  Statistical  Tests 
A.  Croup  1 

Test  the  hypothesis  that  the  mean  of  each  category  estimated  at  T.,  is 
equal  to  the  mean  of  the  actual  values  at  T,^.  is  the  number  of  SCK's 

completed  by  Group  1.  The  statistical  test  use  is  a  paired  t  test  (1,  p242-246) 
Let  x^j  be  the  estimated  direct  man-days  for  SCR  i  and  cateogry  j 

(1  =  1,2 . njHj  =  1,2,. ..,6) 


Group  1 


Group  2 


IS 


"1^2 


2.  Calculate  S 


»2  -  fc£-|)f  <  'i 


nl+,12 


Calculate  t  =  +  n.; 

~2  v 


(  -S-  V 

\  "i+n2  / 


4.  Compare  to  a  l  distributed  random  variable  with  n  f  n,  -  1 

degrees  of  freedom.  If  |tQ|  too  large,  reject  the  hypothesis 
that  the  mean  of  the  total  estimated  effort  for  both  groups 
is  equal  to  the  mean  of  the  total  actual  effort  expended  by 
the  two  groups. 

D.  Comparison 

To  test  the  effectiveness  of  providing  reliable  estimates  by  esti¬ 
mating  at  T  ,  compare  the  data  with  the  T3  data.  All  the  tests  described 
in  A,  B,  and  C  would  bo  repeated  for  this  different  data  set. 

E .  Scatter  Diagram 

An  additional  technique  for  investigating  visually  the  relationship 
between  the  actual  values  and  the  estimated  value  can  be  accomplished 
through  the  use  of  a  scatter  diagram.  That  is,  plot  the  estimated  value 
for  each  data  point  against  the  actual  value  for  that  point.  An  example 
of  a  scatter  diagram  is  shown  below: 


Estimated 


Actual 
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This  technique  is  useful  in  providing  insight  regarding  the  bias  asso¬ 
ciate  with  data  where  it  lias  been  shown  statistically  that  the  means  are 
not  equal  with  any  statistical  confidence. 

I  .  ,  ,  ,  •  •  >'*!..'• 

VI.  Conclusions  — <  ' 

vV  f'  \  -  4  .*  t  .  /*  .* 

After  having  studied  the  data  gathered  by  Raven  Systems  regarding 

the  use  of  the  existing  Form  50(and  having  visited  Support  Group  Lee 

'  •  .1  ' 

and  observed  how  they  operate,  the  folic 'ing  recommendations  are -'suggested 
These  recommendat ions^are  focused  strictly  on  the  task  of  micro-resource 
ost imat ion  * 

1N,  Form  50,  the  existing  mechanism  for  estimating  resources 
expended,  should  be  replaced  by  an  estimating  methodology 
based  on  the  direct  man-days  required  to  accomplish  the 
task!  The  new  format  for  these  estimates  is  presented 
in  Appendix  B. 

2.  Estimates  should  be  made  by  the  person  or  persons  who 
will  actually  make  the  change  immediately  prior  to  the 
initiation  of  the  work.  If  estimates  must  be  made  some 
time  in  advance,  new  estimates  should  be  made  just  prior 
to  undertaking  the  workj 

3.  The  Project  Management  System  should  be^utilized  to  track 
the  direct  and  indirect  effort  associated  with  each  System 
Change  Request  (SCR).  This  record  keeping  will  ensure  the 
availability  of  data  so  that  improvements  in  the  micro¬ 
estimating  process  will  result. 
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4.  Statistical  verification  of  the  new  methodology  will  be 

accomplished  by  the  study  over  the  period  of  approximately 
one  year  of  actual  vs.  estimated  man-days  required.  Two 
teams  of  Support  Group  Lee,  SAAS  LVL  I  and  A(-)/ACS,  have 
been  selected  to  provide  as  the  test  environment  for  the 
verification  of  the  proposed  procedures.  This  study  will 
provide  the  first  reliable  data  that  should  be  the  basis 
of  any  well  constituted  estimating  procedure. 
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APPENDIX  A 


IMPACT  ESTIMATING 

For  m«  of  this  form,  na  USACSC-SGL  Mtmo  18-1  (Chap  SI;  tha'proponant  aaaocy  It  USAC&C  SptOp,  Ft  Laa  (OAO) 


SYSILM _ 

SCR/PROGRAM 


ESTIMATOR'S  NAMEJS) 
DATE 


T fits  form  is  to  be  used  for  impacting  man-days  effort  required  for  implementation  of  the  above  SCR/program.  Standard 
factors  are  shown  below.  This  form  is  to  be  attached  to  USACSC  Form  6. 


SECTION  1 

1.  INPUT  FILE  FORMATS  AFFECTED  BY  THIS  SCR 

a.  Number  of  card  files  _ 

b.  Number  of  tape  files  _ 

c.  Number  of  disk  files  _ 

Number  X  Factor 

X  1  - 

XI  - 
XI  - 

__  TOTAL 

2.  OUTPUT  FILE  FORMATS  AFFECTED  BY  THIS  SCR 

a.  Number  of  card  files  _ 

XI  - 

b.  Number  of  tape  files 

_  X  1  - 

c.  Number  of  disk  files 

XI- 

d.  Number  of  report  formats 

X  1  - 

TOTAL 

3.  PROGRAM  FUNCTIONS*  NOTE:  This  table  reflects  the  number  of  programs  which  include  functions  effected  by  the 
SCR,  (e  g..  An'scttmay  aflect  an  edit-validation  function  in  each  of  3  programs.  Two  are  simple.' one  is  complex.  Enter: 
a.  Edit  validation  4  X  2  -  8  8X1-8  12X0  =  01 


a.  Edit-validation 

b.  Sort/merge  process 

c.  Interns I  data  manipulation 
1  d.  File  search 

.e.  Table  look-up  (internal  or  external) 

-I.  Calculations 

■fl.  Utilities  or  subroutines 

‘h.  Job  Control  languages 

Subtotals 

r  Total  of  Program  Functions 


SIMPLE 
Factor  X  Pgms 
4  X  - 


COMPLEX 
Factor  X  Pgms 

8  X  - 

3  X - 

3  X - 

3  X  _ 

5  X _ 

5  X _ 

3  X _ _ 

2  X  _ 


VERY  COMPLEX 
Factor  X  Pgms 
12  X  - -  - 


USACSC- FT  LEE  FUHM  bU 

1  MAR  77 


8.  SYSTEM  FACTOR 


A  * . 


7.  PROGRAM  TURN  AROUND  TIME  (Average) 

FACTOR 

*.  Effective  IAP  Usage  0.6 

(  .  More  than  once  per  day  0.8 

c.  Once  per  day  1.0 

d.  Less  than  once  per  day.  1.2 


9.  DOCUMENTATION  CHANGES  REQUIRED  BYTHIS  SCR 


Number  of  pages  to  be  chanqed/added 


a.  Developmental  ~ 

b.  Major  change 

c.  Major  modification 

d.  Minor  modification 

e.  Maintenance 

f.  Minor  technical  change 
Q.  JCL  change  only 


FACTOR 

2 

3 

4 

5 

6 

7 

8 


10.  COMPUTER  TIME  REQUIRED 
Hours  * 


(Total  of  Column  13  is  entered  onto  Line  #1  of  the  SCR  Estimate  Summary  and  will  be  defined  as  Net  Development  Time 
Ion  the  SCR  Estimate  Summary. 


1.  (1) 
Input  Total 


Total  from 
(3a) 


Total  from 
<7e) 


(2)  (3) 

Output  Total  Program 

Function  Total 

♦  + 


SECTION  II 

NET  DEVELOPMENT  TIME 
.  (3a) 

Sub-Total  NOTE:  If  (3a)  is  zero,  enter  one. 


(4) 

Resources 

Average 


(8) 

System 

Factor 


(6) 

Job  Knowledge 
Required 


(6)  (7) 

Job  Knowledge  Program  Turn- 

Available  around  Factor 

X  X 


(7a) 

Sub- total 


(9) 

Development 

Time 


X  2  - 


(10) 

Other  System 
Factor 

1.8 


(ID  (12) 

Non-Project  Lost  Time 
Factor  Factor 

X  (1.25  ♦  0.1) 

2.43  man-days 


(13) 

Net  Development 
Time 


SCR  ESTIMATE  SUMMARY 

I.  Net  Development  Time  “  man  days 

a.  Review  and  analysis  -  “NOT  X  0.16  “ 

X 

8- 

• 

lb.  Design 

-  NOT 

X  0.20  - 

X 

8- 

• 

*.  Programing  (including  Level  1  testing) 

-NOT 

X  0.35  - 

X 

8- 

• 

•<L  Testing  (including  Level  II  &  III  testing) 

-NDT 

X  0.25  - 

X 

8- 

• 

it.  Documentation  (enter  2ero  for  none) 

*  . 

-  NDT 

X  0.05  - 

X 

8  - 

• 

2.  Total  project  man-days  (sum  of  la-e  above) 

a 

TPMD 

man-days  X 

8- 

man-hourj 

*  Enter  these  figures  in  the  appropriate  blocks  of  the  Impact  Analysis  section  of  USACSC  Form  6.  (System  Change 
Request) 
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SYSTEM 
SCR  NUMBER 


APPENDIX  B 

DIRECT  MAN  HOUR  ESTIMATING 
DATE  _ _ _ 

_  ESTIMATOR’S  NAME  _ 

PROGRAM 


R-l 


1.  Review  and  Analysis  _  Man-hrs. 

(Includes  research,  clarifying  change  request, 
interacting  with  proponent  agency) 

(i.e.  What  is  to  be  done) 


2.  Design  _  Man-hrs. 

(Problem  definition  at  program  level,  synthesis, 
develop  change  specif ication) 

(i.e.  How  it  is  done) 

3.  Communicat ion  _  Man-hrs. 

(Interaction  between  programmer  and  systems 
analyst ) 


4.  Coding  Man-hrs. 

(All  coding  activities  to  implement  change 
specif ications) 

i 

5.  Level  I  Testing  Man-hrs. 

(Compilation,  development  of  test  data, 
analysis  of  test  results  and  corrections) 

Estimated  Number  of  Runs  _ _ 

6.  Documentation  Man-hrs.  I 

(Any  documentation  required  for  completed  \ 

change) 


TOTAL 


Man-hrs . 


NARRATIVE 


1.  Programs  Affected 


APPENDIX  C 


C-l 


DIRECT  MAN  HOUR  ESTIMATING 
DATE 

■O  '■  I  I.M  ESTIMATOR’S  NAME 

'■OR  NUMBER  _  PROGRAM 


Review  and  Analysis 

Research 

Man-hrs . 

Interacting  with  proponent  agency 

Man-hrs . 

lies  i  gn 

Problem  definition 

TOTAL  _ 

Man-hrs . 

Man-hrs 

Sy lit  lies  i  s 

Man-hrs . 

Develop  specification 

Man-hrs . 

TOTAL 

Man-hrs 

Common i cat  ion 

TOTAL 

Man-hrs 

4.  Coding 


Program  Complexity  j 

Extent  of  Change 

Simple 

Normal 

Complex 

Simple 

Minor 
Mod . 

Major 

Mod. 

Dev. 

a  .  I/O  Korina  t  s 

1 

0 

3 

1 

;  2 

3  1 

!  4 

Man-hrs 

h.  Ed  it /Valid. 

1 

2 

3 

* 

2 

3 

j  4 

Man-hrs 

e .  1 nterna I 

Process  ing 

1 

2 

3 

1 

1 

2 

3 

4 

_ _ _ _  Man-hrs 

d.  Internal  Data 

Man i pn 1  at  ion 

! 

2 

3 

1  S 

2 

3 

,  4 

Man-hrs 

e  .  i  ’t  iie  r 

i 

j 

• 

1 

. 

Man-hrs 

TOTAL _ _ 

___  Man-hrs 

Love  1  I  Test  ; ng 

a .  Comp i lat ion/Ver icat ion 

Man-hrs . 

h.  Development  of  test  data 

Man-hrs. 

e.  Validation  of  test  results 

Man-hrs . 

TOTAL 

Man-hrs 

Doc timeiu.it  ion 

a  .  U.'.<  r  Manna  1 

Man-hrs . 

l>.  Oper. it  ions  A  Scheduling  Manual 

Man-hrs. 

e.  Program  logic  documentation 

_ _ _ __  Man-hrs. 

TOTAL 

_  Man-hrs 

TOTAL  MAN-HRS.  TO  COMPLETE  CHANGE 

Man-hrs 

C-2 


NARRATIVE 


1.  Programs  Affected 


2.  Files  Affected 


3.  Description  of  program  changes  on  which  this  estimate  is  based. 


/ 


END 


DATE 
FILMED 


DTIC 


